Analysis of Integrated Data without Data Integration

M scientific and policy investigations require statistical analyses that “integrate” data stored in multiple, distributed databases. For example, a regression analysis on integrated state databases about factors influencing student performance would be more insightful than individual analyses, or complementary to them. Other contexts where the same need arises range from homeland security to environmental monitoring. At the same time, the barriers to actually integrating the databases are numerous. One is confidentiality: the database holders—we term them “agencies”—almost always wish to protect the identities of their data subjects. Another is regulation: the agencies may be forbidden by law to share their data, either with each other or with a trusted third party. A third is scale: despite advances in networking technology, the only way to move a terabyte of data from point A today to point B tomorrow is FedEx. The good news is that for many analyses it is not necessary to move the data. Instead, using techniques from computer science known generically as secure multiparty computation, the agencies can share summaries of the data anonymously, but in a way that the analysis can be performed in a statistically principled manner. In this article we illustrate linear regression on “horizontally partitioned data.” Only one concept is needed, that of secure summation, which is shown in Figure 1. There are other approaches to this problem for lower risk situations, as How can secure multiparty computation enable agencies to share information without sacrificing confidentiality?